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SUBJECT:  Statistical  Decisions  Utilizing  Neural  Nets. 

1 .  Summ.arv : 

Neural  networks  were  developed  that  accurately  determine  the  statistical 
characteristics:  modality  and  number  of  stochastic  components  of  underlying 
probability  distributionfs)  for  sample  data.  Sample  data  examples,  used  to 
teach  the  neural  nets  were  generated  utilizing  either  a  single  beta  distri¬ 
bution  or  a  mixture  of  beta  distributions.  Once  the  neural  net  learned  to 
distinguish  between  unimodal  and  multimodal  examples  and  also  between  unimodal 
and  mixture  densities,  they  were  challenged  with  unknown  test  cases.  The  test 
cases  were  also  generated  from  either  a  single  beta  distribution  or  a  mixture 
of  beta  distributions.  Therefore  the  initial  test  results  apply  to  a 
restricted  class  of  distributions  having  bounded  domains .  However . these 
trained  networks  were  furnished  to  Mr.  Schlenker  who  challenged  the  networks 
with  sample  data  from  distributions  other  than  beta,  thus  widening  the  appli¬ 
cation  domain  of  the  networks.  An  explanation  of  this  additional  work  is 
detailed  in  appendix  A. 

The  initial  testing  of  the  neural  networks  consisted  of  40  unknown  sample 
data  examples  generated  utilizing  beta  distributions.  The  results  of  these 
tests  are:  (1)  correctly  identified  39  out  of  the  40  as  being  either 
unimodal  or  multimodal,  an  accuracy  of  97.5  percent.  This  exceeds  the 
accuracy  of  currently  available  statistical  methods;  (2)  correctly 
identified  36  out  of  40  as  having  either  one  component  or  more  than  one 
component,  an  accuracy  of  90  percent. wThere  are  no  statistical  methods 
available  for  determination  of  components.  For  the  larger  class  of 
distributions,  the  corresponding  accuracy  rates  are  93  percent  and  81  percent. 

Mr.  Manata's  work  in  developing  the  neural  networks  was  originally 
published  in  memorandum  report  SA-MR-9002.  Mr.  Schlenker 's  work  on  the 
extensions  was  originally  published  in  memorandum  report  SA-MH-9003.  But, 
since  these  two  reports  are  interrelated,  they  are  being  consolidated  in  this 
report . 

2.  Neur.il  Network  Discussion: 


a.  Knowledge: 


There  are  at  least  three  ways  of  representing  knowledge  in  a  computer 
environment:  staniard  computer  programs,  expert  systems,  and  neural  networks. 

A  standard  program  has  two  types  of  knowledge:  instructions,  and  the 
value  of  the  variables  used  by  the  program  If  a  user  wants  to  know  what 
knowledge  the  computer  contains,  the  list  of  instructions  and  the  current 
value  of  the  variables  provides  this  information. 


An  expert  system  has  three 
i2)  initial  facts  and  beliefs,  (3) 

by  listing  the  current  rule,  facts 
that  the  system  uses. 


types  of  knowledge:  (1)  if  -  then  rules, 
conclusions  generated  by  the  if  -  then 
a  us  or  cCi*  dctcrinins  the  sySvcnio  Xn 
b''  iefs,  and  the  value  of  any  variables 
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Neural  networks  contain  two  types  of  knowledge:  the  network,  nodes 
and  the  connections  between  nodes,  and  the  weights  associated  with  each 
connection.  If  the  user  has  a  question  concerning  the  computer’s  knowledge, 
the  network  and  the  weights  are  available.  But,  the  user  would  not  find  it 
easy  to  ascertain  what  this  knowledge  has  to  do  with  solving  the  problem.  As 
an  example  of  neural  network  knowledge,  figure  1.0  shows  the  type  of  network 
that  was  used  to  solve  the  statistical  decision  problem,  and  table  1.0  is  a 
partial  listing  of  the  weight  values. 


FIGURE  1.0 

NETWORK  ARCHITECTURE 


TABLE  1.0 

CONNECTION  WEIGHTS 
MIDDLE  LAYER  NODES 


INPUT 

LAYER 

NODES 

1 

2 

3 

1 

-.  175 

-  .890 

-.263 

2 

-.479 

♦  .415 

-2.177 

3 

-1.117 

-1.003 

+6.535 

4 

-1.123 

+  .282 

-7.678 

MIDDLE 

LAYER 

NODES 

OUTPUT  LAYER 

1 

NODE 

1 

+  .880 

2 

-.849 

3 

+19.521 

2 


b.  Neural  Networks: 


Neural  networks,  of  the  type  shown  in  figure  1.0,  have  three  or  more 
layers  of  nodes  with  the  nodes  of  one  layer  connected  to  all  the  nodes  of  the 
next  layer.  Neural  network  problem  solving  is  initiated  by  providing  the 
first  layer,  the  input  layer,  with  a  vector  of  values,  the  input  vector, 
containing  information  which  describes  the  problem  to  be  solved.  Each  input 
node  receives  one  component  of  this  vector  and  feeds  it  into  the  connections 
between  itself  and  every  middle  layer  node.  The  connections  multiply  these 
components,  by  the  connection  weights,  and  deliver  the  modified  component 
values  to  the  middle  layer  nodes.  Each  middle  layer  node  sums  ivs  incoming 
values  and  operates  on  the  sum  with  an  activation  function,  usually  a  sigmoid 
function.  This  generates  a  nodal  output  value  which  is  furnished  to  the 
connections  between  the  middle  layer  nodes  and  every  output  node.  The  multi¬ 
plication  process  that  occurred  between  the  input  layer  and  middle  layer  is 
repeated  between  the  middle  and  output  layers.  The  output  nodes  sum  the 
incoming  values,  operate  on  the  sum  with  the  activation  function,  and  generate 
output  values.  These  output  values  are  the  answer  to  the  the  problem. 

Before  a  network  can  solve  a  specific  problem  it  has  to  be  taught  how 
to  solve  the  set  of  problems  of  which  the  specific  problem  is  a  member.  This 
requires  that  the  network  be  furnished  representative  examples  of  the  problem 
set,  and  the  answer  for  each  example.  During  training,  the  network  compares 
its  answer  with  the  correct  answer.  If  its  answer  is  within  a  specified  range 
of  the  correct  answer,  the  network  is  considered  to  have  ‘learned’  to  solve 
the  set  of  problems.  This,  delta  value,  set  prior  to  training,  is  the  maximum 
error,  between  the  correct  and  network  answers,  that  the  developer  will 
accept.  Delta  is  usually  set  at  0.1  but  it  can  be  any  value  greater  than  zero 
and  less  than  one.  Until  this  criterion  is  met,  the  network  back  propagates 
the  actual  error  through  the  network  to  modify  the  connection  weights.  This 
feed-forward-back-propagation  process  continues  until  the  network  has  achieved 
‘learning’  as  defined  by  convergence  to  a  given  limit.  Once  the  network  has 
‘learned,’  it  is  tested  to  determine  its  accuracy. 

c.  Neural  Net  Simulation: 

Neural  networks  are  a  parallel  processing  technique.  But,  they  can  be 
emulated  on  sequential  computers.  This  is  accomplished  by  utilizing  simula¬ 
tion  software.  AMSMC-SAO  has  simulation  software  (NETS)  developed  by  NASA  for 
IBM-compatible  PCs. 

d.  Statistical  Decision  Making: 

Statistical  aids  utilizing  neural  networks  were  suggested  to  Mr.  J. 
Manata  of  AMSMC-SAO  by  Mr.  G.  Schlenker  of  AMSMC-SAS.  The  statistical 
questions  concerned  the  modality,  and  the  number  of  stochastic  components  (the 
number  of  unimodal  components  in  a  probability  mixture  model)  of  probability 
d i s tr ibut/ i ons  producing  S3nipl9  niothods  sxist  foi*  dovsr* 

mining  modality  but  not  for  number  of  components. 
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3.  Methodology: 

a.  Net  Architecture: 

The  sample  data  was  distributed  over  20  histogram  cells  anticipating 
that  the  neural  networks  would  be  used  with  samples  of  200  or  more  data 
points:  therefore,  20  cells  seemed  a  reasonable  number  to  assure  an  adequate 
population  for  each  cell.  The  number  of  output  nodes  was  set  at  either  one  or 
two  depending  on  the  type  of  problem  the  network  was  required  to  solve.  If 
the  network  was  required  to  solve  a  modality  or  component  problem,  the  number 
of  outputs  was  one.  If  the  network  was  required  to  solve  two  problems, 
modality  and  components,  the  number  of  outputs  was  two.  The  number  of  middle 
layer  nodes  and  the  connections  between  the  input  layer,  the  middle-layer, 
and  the  output  layer  was  determined  by  trial  and  error.  Nineteen  net  archi¬ 
tectures  were  tested.  Table  2.0  lists  the  different  architectures,  and  figure 
2.0  shows  examples  of  the  architectures. 

The  architecture  that  worked  the  best  had  twenty  input  nodes,  seven 
middle  nodes,  and  one  or  two  output  nodes.  An  example  of  this  architecture  is 
shown  in  figure  2b. 


NUMBER  OF 
INPUT  NODES 

NUMBER  OF 
MIDDLE  LAYERS 

TOTAL  NUMBER 
OF  MIDDLE 
LAYER  NODES 

NUMBER  OF 
MIDDLE  LAYER 
NODES  PER 
MIDDLE  LAYER 

NUMBER 

OUTPUT 

OF 

NODES 

FIGURE 

20 

0 

0 

0 

2 

8a 

20 

1 

3 

3 

2 

8b 

20 

1 

4 

4 

2 

8b 

20 

1 

5 

5 

2 

8b 

20 

1 

7 

7 

2 

8b 

20 

1 

9 

9 

2 

8b 

20 

1 

10 

10 

2 

8b 

20 

1 

11 

11 

2 

8b 

20 

1 

12 

12 

2 

8b 

20 

2 

7 

4.3 

2 

8c 

20 

2 

7 

3.4 

2 

8c 

20 

2 

7 

4,3 

2 

8d 

20 

2 

7 

3.4 

2 

8d 

20 

2 

7 

4.3 

2 

8e 

20 

2 

7 

5.2 

2 

8c 

20 

2 

7 

2.5 

2 

8c 

20 

2 

7 

5.2 

2 

8e 

TABLE  2.0 

LISTING  OF  NETWORK  ARCHITECTURES  ATTEMPTED 


FIGURE  2.0 

EXAMPLES  OF  NETWORK  ARCHITECTURE 


b.  Generation  of  Training  and  Test  Data  Sets: 

The  training  sample  data  distributions,  generated  by  Mr.  Schlenker, 
were  developed  by  a  Monte-Carlo  selection  of  random  variables  from  beta 
distributions  or  mixtures  of  beta  distributions  [a].  Beta  distributions  were 
used  because  they  provide  the  capability  for  the  generation  of  a  variety  of 
density  function  shapes,  and  they  have  a  well-defined  finite  domain. 

Each  sample  consisted  of  400  data  points  which  were  distributed  over 
20  histogram  cells.  Two  separate  training  sets  were  generated;  one  set  had 
examples  containing  sampling  error,  noise,  and  the  other,  an  idealized  set, 
did  not.  The  localized  case  represents  an  essentially  infinite  sample.  The 
use  of  two  training  sets  was  to  determine  if  the  network  was  more  accurate 
when  trained  with  noisy  or  idealized  data. 

Mr.  Schlenker  also  generated  test  examples  utilizing  one  or  more  beta 
distributions,  plus  one  example  from  a  truncated  weibull  distribution.  The 
number  of  data  points  in  the  test  examples  varied  between  150  and  3200.  Table 
3.0  lists  the  training  sets  and  table  4.0  lists  the  test  sets.  Table  5.0 
shows  the  location  of  the  modes  and  provides  a  relative  measure  of  their 
separation  in  terms  of  the  standard  deviation.  Additionally,  the  coefficient 
of  skewness  (beta)  of  the  distribution  is  shown  to  suggest  the  range  of  this 
parameter  that  the  network  had  to  recognize.  With  the  exception  of  seven 
sets,  all  the  sets  were  generated  using  a  Simscript  random  number  seed  index 
1. 


Table  6.0  lists  the  parameter  values  of  the  beta  mixture  which  produce 
each  set.  For  the  unimodal,  one-component  sets,  the  threshold  parameter  is 
always  0,  and  the  upper  limit  is  1;  i.e.,  the  distribution  form  is  standard¬ 
ized  beta.  In  the  case  of  two  stochastic  components,  either  one  or  two  modes 
are  produced,  depending  on  the  parameters  of  each  stochastic  component  and  on 
the  mixture  parameter  (r) .  In  all  cases,  save  two,  the  threshold  and  limit 
parameters  are  0  and  0.7,  for  component  1,  and  0,3  and  1.0  for  component  2. 

The  two  exceptions  are  sets  HM36  and  HM41.  For  HM36  these  parameters  are  (0, 
0.5),  for  component  1,  and  (0.3,  1.0)  for  component  2.  For  HM41  these  para¬ 
meters  are  (0.  0.6),  for  component  1,  and  (0.4,  1.0)  for  component  2.  As  is 
Seen,  the  lower  and  upper  limits  on  the  domain  of  all  mixtures  are  0  and  1. 
These  are  also  the  respective  histogram  limits. 


Ca]  The  density  function  for  a  mixture  of  beta  densities  is  given  for  the 
random  variable  x  as 

Al-1  Bl-1  A2-1  B2-1 

f(x)  =  rw  (1-w)  /CB(Al.Bl)  +  (l-r)z  (1-z)  /CB(A2,B2)  , 

where  r  is  proportion  of  first  component  and  where  CB(a,b)  is  the 
complete  beta  function  with  parameters  a  and  b,  and  where  A1 ,  B1  are 
parameters  of  the  first  component  and  A2,  B2  are  parameters  of  the 
second.  Auxiliary  variables  w,  z  are  given  in  terms  of  x  as 
w  =  (x  -  Thl)/(U11  -  Thl)  and  z  =  (x  -  Th2)/(U12  -  Th2)  ,  for  w  and  z 
limited  to  the  unit  interval.  Threshold  and  upper  limit  parameters  for 
component  c  are  The  and  Ulc. 
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c.  Scaling  of  Nodal  Input  Values: 

A  requirement  of  the  NETS  is  that  the  nodal  input  values  lie  in  the 
range  of  (.1,.9).  Because  of  this  requirement,  the  histogram  of  the  training 
and  test  examples  had  to  be  modified  to  lie  within  this  range.  This  scaling 
was  accomplished  by: 

e  =  n/N  +  0.1 

where 

e  =  scaled  value 
N  =  sample  size 
n  =  interval  frequency 


TABLE  3.0 
TRAINING  SETS 


DISTRIBUTION  USED 
TO  GENERATE 
TRAINING  SAMPLE 

NUMBER  OF  DATA 
POINTS  IN  EACH 
SAMPLE 

SAMPLE 

DISTRIBUTION 

STOCHASTIC 

COMPONENTS 

NUMBER  OF 
TRAINING  SETS 

ONE  BETA 

400 

UNI MODAL 

1 

30 

TWO  BETAS 

400 

UNI MODAL 

2 

9 

TWO  BETAS 

400 

MULTIMODAL 

2 

23 

TABLE  4.0 
TEST  SETS 


DISTRIBUTION  USED 
TO  GENERATE 

TEST  SAMPLE 

NUMBER  OF  DATA 
POINTS  IS  EACH 
SAMPLE 

SAMPLE 

DISTRIBUTION 

STOCHASTIC 

COMPONENTS 

NUMBER  OF 

TEST  SETS 

ONE  BETA 

150 

UNI MODAL 

1 

4 

ONE  BETA 

200 

UNI MODAL 

1 

2 

ONE  BETA 

400 

UNI MODAL 

1 

13 

ONE  WEIBULL 

3200 

UNI MODAL 

1 

1 

TWO  BETAS 

400 

UNI MODAL 

2 

9 

TWO  BETAS 

3200 

UNI MODAL 

2 

2 

TWO  BETAS 

400 

MULTIMODAL 

2 

9 
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TABLE  5.0  Training  Sets  for  Networks  to  Classify  Samples  of  Density 

Functions  with  Respect  to  Modality  and  Number  of  Components 


Set  Name 

Modes  Components 

Loc  Modes 

Std  Dev 

Skewness  Seed 

HUOl 

uni modal 

1 

0.00, 

0.200 

1.200  1 

03 

0.34, 

0.200 

0.286 

05 

0.66, 

---- 

0.200 

-0.286 

09 

0.00, 

— 

0.194 

0.861 

13 

0.34, 

---- 

0.200 

0.286  3 

15 

0.66, 

---- 

0.200 

-0.286 

19 

0.00, 

---- 

0.194 

0.861 

22 

0.25, 

---- 

0.140 

0.488 

24 

0.50, 

---- 

0.140 

0.000 

35 

0,50, 

---- 

0.121 

0.000 

37 

0.70, 

0.131 

-0.364 

HMOe 

bimodal 

2 

0.35, 

0.65 

0.179 

0.000  1 

11 

0.35, 

0.65 

0.176 

0.237 

16 

0.35, 

0.65 

0.176 

-0.237 

26 

0.35, 

0.65 

0.168 

-0.476 

32 

0.30, 

0.65 

0.197 

0.486 

34 

0.30, 

0.65 

0.198 

0.247 

36 

0.25, 

0.65 

0.222 

0.242 

38 

0.37. 

0.65 

0.184 

0.150 

41 

0.30, 
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0.214 

0.000 
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HMIO 

uni modal 

2 
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0.217 

0.000 

14 

0.45, 
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0.168 

15 

0.48, 

— 

0.215 

0.131 

18 

0.62, 

0.188 

-0.196 

20 

0.53. 

0.215 

-0.131 

22 

0.35, 

---- 

0.173 

0.436 

25 

0.44, 

— 

0.208 

0.251 

27 

0.65, 

.... 

0.173 

-0.436 

30 

0.56, 

•  *  “  * 

0.208 
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TABLE  6.0  Parameters  Which  Characterise  the  Probability 
Distributions  Used  in  Producing  Training  Sets 


Set 

Name 

Mix 

Param 

Component  1 

A  B 

Component  2 

A  B 

HUOl 

— 

0.6 

2.4 

03 

--- 

2.0 

3.0 

05 

... 

3.0 

2.0 

09 

... 

1.0 

3.0 

13 

— 

2.0 

3.0 

15 

... 

3.0 

2.0 

19 

— 

1.0 

3.0 

22 

— 

2.914 

6.8 

24 

— 

5.878 

5.878 

35 

... 

8.0 

8.0 

37 

— 

8.0 

4.0 

HM06 

0.5 

6.0 

6.0 

6.0 

6.0 

1 1 

0.6 

6.0 

6.0 

6.0 

6.0 

16 

0.4 

6.0 

6.0 

6.0 

6.0 

26 

0.3 

6.0 

6.0 

6.0 

6.0 

32 

0.6 

6.0 

6.0 

4.0 

4.0 

34 

0.6 

4.0 

4.0 

6.0 

6.0 

36 

0.5 

6.0 

6.0 

4.0 

4.0 

38 

0.5 

6.0 

6.0 

4.0 

4.0 

41 

0.5 

7.0 

7.0 

7.0 

7.0 

44 

0.6 

7.0 

7.0 

7.0 

7.0 

HMIO 

0.5 

2.0 

2.0 

2.0 

2.0 

14 

0.6 

3.0 

3.0 

3.0 

3.0 

15 

0.6 

2.0 

2.0 

2.0 

2.0 

18 

0.4 

4.0 

4.0 

4.0 

4.0 

20 

0.4 

2.0 

2.0 

2.0 

2.0 

22 

0.7 

5.0 

5.0 

5.0 

5.0 

25 

0.7 

2.0 

2.0 

2.0 

2.0 

27 

0.3 

5.0 

5.0 

5.0 

5.0 

30 

0.3 

2.0 

2.0 

2.0 

2.0 

d.  Training  Sets: 

The  next  step  in  the  process  of  developing  an  accurate  neural  net,  is 
the  determination  of  the  best  set  of  training  examples.  The  best  set  of 
training  examples  is  that  combination  of  examples,  which  results  in  a  trained 
net,  that  provides  the  highest  accuracy  when  the  net  is  challenged  with 
unknown  test  cases. 

Examples  of  the  training  sets  are  shown  in  figures  3.0  -  8.0,  figure 
3.0  is  the  histogram  for  a  unimodal  distribution  with  one  stochastic  com¬ 
ponent;  figure  4.0  is  the  theoretical  density  for  the  sample;  figure  5.0  is 
the  histogram  for  a  unimodal  distribution  with  two  stochastic  components; 
figure  6.0  is  ti...  theoretical  density  for  the  sample;  figure  7.0  is  the 
histogram  for  a  multimodal  distribution;  and  figure  8.0  is  the  theoretical 
density  for  the  sample. 
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FIGURE  4.0 
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FIGURE  6.0 

THEORETICAL  DISTRIBUTION 
UNIMODAL  DENSITY  WITH 
TWO  STOCHASTIC  COMPONENTS 
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4.  Availability  of  Neural  Networks: 


Thera  are  three  ways  of  obtaining  working  versions  of  the  statistical  aids 
neural  networks: 

a.  NETS  version  2  is  capable  of  generating  delivery  versions  that  can 
be  installed  on  PCs. 

b.  The  networks  can  be  implemented  in  standard  computer  programs 
written  in  high-level  languages. 

c.  The  neural  networks  have  been  incorporated  into  a  ‘statistical 
expert  system'  (DESCR. STATS)  that  is  available  on  the  PRIME  computer. 

The  incorporation  of  the  neural  networks  Into  the  ‘statistical  expert 
system,"  is  explained  in  appendix  A  and  a  listing  of  the  computer  program  is 
shown  in  appendix  B. 

5.  Results: 

a.  Modality: 

The  training  set  for  modality,  that  furnished  the  most  accurate  test 
results,  consisted  of:  twelve  unimodal  examples  with  one  stochastic  compo¬ 
nent,  nine  unimodal  examples  with  two  stochastic  components,  and  ten  multimodal 
examples.  This  network,  when  challenged  with  unknown  beta  distribution  test 
cases,  correctly  identified  39  out  of  40  cases,  an  accuracy  of  97.5  percent. 
This  same  network,  when  challenged  with  test  cases  generated  using  other 
distributions,  had  an  accuracy  of  91  percent.  These  accuracies  exceed  the 
accuracy  of  the  current  statistical  method  for  determining  modality  of  sample 
data  distributions.  The  accuracy  of  the  network,  trained  with  idealized 
sample  data  which  was  tested  using  noisy  beta  distribution  data,  was  identical 
to  that  trained  on  the  noisy  data. 

b.  Number  of  Stochastic  Components: 

The  training  set  of  stochastic  components,  that  furnished  the  most 
accurate  test  results  consisted  of:  thirteen  unimodal  examples  with  one 
stochastic  component  and  nine  unimodal  examples  with  two  components.  This 
network,  when  challenged  with  unknown  beta  distribution  test  cases,  correctly 
identified  36  out  of  40  cases,  an  accuracy  of  90  percent.  Two  of  the  errors 
were  the  designation  of  one  stochastic  component  as  two,  and  the  other  two 
errors  were  the  designation  of  two  stochastic  components  as  one.  The  same 
network,  when  challenged  with  test  cases  generated  using  other  distributions, 
had  an  accuracy  of  81  percent.  The  network  trained  on  idealized  data,  when 
tested  with  noisy  beta  distribution  data,  was  less  accurate  than  the  one 
trained  on  noisy  data. 


16 


Combined  Modality  and  Components: 


c , 


k  network  was  trained  to  furnish  the  combined  answers,  modality  and 
number  of  components  when  presented  with  a  set  of  unknown  data.  The  best 
network,  trained  on  noisy  data,  furnished  correct  answers  for  modality  in  37 
out  of  40  tests;  for  number  of  components  it  furnished  correct  answers  in  35 
out  of  40  tests.  The  network  did  not  furnish  any  combined  errors,  a  wrong 
answer  for  modality  combined  with  a  wrong  answer  :'or  number  of  components. 

The  network  trained  with  idealized  data  gave  the  same  number  of  incorrect 
answers  for  modality,  3  out  of  40,  but  it  also  furnished  14  incorrect  answers 
out  of  40  tests  for  number  of  components.  This  network  did  not  furnish  any 
combined  errors. 

d.  Number  of  Data  Points. 

The  number  of  data  points  in  the  test  sample  distributions  was  varied 
between  150  and  3200.  Within  this  range  the  accuracy  of  the  neural  networks 
did  not  appear  to  depend  on  sample  size. 

e.  Availability  of  Neural  Networks: 

The  neural  networks  developed  for  modality  and  components  are 
available  on  the  PRIME  computer.  It  is  also  possible  to  develop  delivery 
copies  of  the  networks  for  use  on  PCs. 

6.  Conclusions: 

a.  The  neural  networks  developed  using  NETS  are  capable  of  accurately 
estimating  modality  and  stochastic  components  from  sample  data. 

b.  The  number  of  data  points  in  a  sample  can  be  as  small  as  150. 

c.  Neural  networks  that  furnish  just  one  statistical  attribute  were  more 
accurate  than  the  one  that  furnished  more  than  one. 

d.  Currently  the  modality  network  and  the  component  network  are  available 
on  the  PRIME  computer. 
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APPENDIX  A 


Implementation  of  Neural  Networks 
Into  An  Expert  System 


A-1 


1 .  Implementation  of  Neural  Networka  into  an  ES 


A  major  effort  was  required  to  implement  neural  networks  for  identifying 
univariate  data  features  into  the  statistical  expert  system  DESCR. STATS.  A 
stand-alone  program  (RUN.1D.NET)  was  written  to  verify  and  validate  the  net¬ 
works  for  identifying  modality  and  number  of  stochastic  components.  Mr. 
Schlenker  discovered  that  the  error  rates  of  both  networks  are  quite  sensitive 
to  the  histogram  limits  for  the  data.  Of  course,  this  is  not  a  problem  for 
distributions,  such  as  beta,  whose  domain  is  well  defined.  However,  for 
distributions  with  infinite  or  semi-infinite  (bounded  on  one  side)  domains, 
assigning  proper  limits  for  the  neural  network  is  a  nontrivial  problem.  After 
a  bit  of  computational  experience,  he  found  some  heuristics  for  calculating 
upper  and  lower  limits  which  work  well  with  both  unimodal,  one-component  data 
and  bimodal  data.  Of  interest  is  the  fact  that  the  error  rates  for  types  of 
distributions  such  as  normal,  logistic,  Weibull,  and  lognormal  are  not  much 
greater  than  for  the  beta  (and  mixtures  of  betas) ,  which  was  used  exclusively 
in  training  the  networks.  The  algorithm  for  assigning  histogram  limits  makes 
use  of  the  following  sample  statistics:  average,  standard  deviation,  minimum, 
maximum,  and  the  first  and  third  quartiles. 

2.  AlfSorithm  for  Histojjram  Limits  for  Neural  Networks 

The  method  for  calculating  the  lower  (XHMIN)  and  upper  (XHMAX)  limits  of 
the  histogram  is  sketched  as  follows: 

If  the  random  variable  (X)  represents  a  proportion  or  probability, 

XHMIN  =  0  and  XHMAX  =  1. 

Otherwise,  if  X  is  strictly  positive  (but  not  a  proportion),  the  lower  limit 
is  obtained  by 

XHMIN=max(0,  AVGX  -  3  STDV) ,  if  AVGX  -  4.5  SDV  <  0, 

where  AVGX  and  STDV  are  the  sample  average  and  standard  deviation.  When  this 
condition  does  not  hold,  XHMIN  is  calculated  as  shown  below.  The  upper  histo¬ 
gram  limit  for  this  case  is  calculated  in  the  same  manner  as  that  shown  below 
for  the  case  in  which  X  may  not  be  positive.  If  X  is  effectively  unbounded 
or,  possibly,  capable  of  taking  negative  values,  the  lower  and  upper  limits 
are  calculated  as  follows. 

XHMIN  =  max(XHMIN0,  XHMINl ,  XHMIN2) 


and 


XHMAX  =  min(XHMAX0,  XHMAX 1,  XHMAX2) , 

where  the  alternatives  XHMINO.  XHMINl,  etc.  are  given  in  terms  of  the  minimum 
(XMIN)  and  maximum  (XMAX)  of  the  sample  as  well  as  the  Sample  average  (AVGX) , 
standard  deviation  (STDV),  lower  quartile  (X.25) ,  and  upper  quartile  (X.75) . 


XHMINO  =  AVGX  -  4.5  STDV,  if  this  value  <  XMIN. 


Otherwise , 

XHMINO  =  XHMINO  -  l.KXHMINO  -  XMIN). 

Always 

XHMINl  =  XMIN  -  0.5  STDV 

XHMIN2  =  XMIN  -  0.2(X.25  -  XMIN). 

The  alternative  upper  limits  are 

XHMAXO  =  AVGX  ^4.5  STDV,  if  this  value  >  XMAX. 

Otherwise , 

XHMAXO  =  XHMAXO  ^  1 . 1 (XMAX  -  XHMAXO). 

Always 

XHMAXl  =  XMAX  ^0.5  STDV 

XHMAX2  =  XMAX  +  0.2 (XMAX  -  X.75). 

This  algorithm  produces  histogram  limits  which  conservatively  bound  the  values 
XMIN  and  XMAX. 

3.  Testing  of  the  Neural  Networks 

Two  networks  with  the  same  architecture,  but  different  weights,  are  used 
to  obtain  (a)  an  indication  of  multimodality  and  (b)  an  indication  that  the 
random  variable  comes  from  a  mixture  model.  These  two  attributes  of  each 
random  variable  tested  are  displayed  in  tables  A-1  and  A-2.  If  the  output 
node  for  attribute  (a)  is  <  or  equal  to  (le)  0.5,  the  data  are  declared 
unimodal;  otherwise,  they  are  multimodal.  If  the  output  node  for  attribute 
(b)  is  less  than  0.5,  only  one  stochastic  component  is  identified.  In  these 
tables,  respective  identification  of  attributes  is  designated  c,  for  correct, 
and  «,  for  incorrect.  Histograms  do  not  always  display  the  same  attributes  as 
the  population  density.  For  example,  a  random  variable  with  two  modes  in  the 
density  may  have  a  clearly  unimodal  histogram.  In  all  cases,  errors  in 
modality  and  stochastic  components  are  declared  if  the  population  density  is 
not  correctly  identified.  Three  sample  sizes — 800,  400,  200 — are  used  for 
each  random  number  seed  and  for  each  set  of  population  parameters.  In  most 
cases,  if  the  results  for  a  sample  of  800  are  correct,  the  results  for  the 
smaller  samples  are  as  well.  Mixtures  of  normal  and  logistic  random  variables 
are  found  to  be  classified  nearly  as  well  as  beta  mixtures  on  which  the 
networks  were  trained. 
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TABLE  A-1.  Results  of  Numerical  Experiments  .'n  Testjng  Neural 
Networks  With  Unimodal,  One-Component  Distributions 

Random  Number  Seed  Indices  1  and  5 

Run  Result  Dist'n  Mean  Std  Dev  Sample  Seed 


1 

c 

c 

lognorra 

0.47 

0.187 

800 

2 

c 

c 

400 

3 

c 

c 

200 

4 

c 

X 

weibull 

800 

5 

c 

c 

400 

6 

c 

c 

200 

7 

c 

X 

gamma 

800 

8 

c 

c 

400 

9 

c 

X 

200 

10 

c 

c 

normal 

800 

11 

Z' 

c 

400 

12 

c 

c 

200 

13 

c 

c 

logistic 

800 

14 

c 

c 

400 

15 

c 

c 

200 

16 

c 

c 

lognorm 

0.47 

0.170 

800 

17 

c 

c 

400 

18 

c 

c 

200 

19 

c 

c 

lognorm 

0.20 

0.170 

800 

20 

c 

c 

400 

21 

c 

c 

200 

22 

c 

c 

lognorm 

0.47 

0.270 

800 

23 

c 

c 

400 

24 

c 

c 

200 

25 

c 

c 

weibull 

800 

26 

c 

c 

400 

27 

c 

c 

200 

28 

c 

c 

beta 

0.20 

0.170 

800 

29 

c 

c 

400 

30 

c 

c 

200 

31 

c 

c 

beta 

0.47 

0.270 

800 

32 

c 

c 

400 

33 

c 

c 

200 

34 

c 

X 

beta 

0.47 

0.170 

800 

35 

c 

X 

400 

36 

X 

X 

200 

37 

c 

c 

beta 

0.67 

0.270 

800 

38 

c 

c 

400 

39 

c 

c 

200 

40 

c 

X 

weibull 

800 

41 

c 

X 

400 

42 

c 

c 

200 

43 

n 

n 

800 

44 

c 

c 

400 

45 

c 

c 

200 

Continued  on  next  page. 
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TABLE  A-1.  Results  of  Numerical  Experiments  in  Testing  Neural 
Networks  With  Unimodal ,  One-Component  Distributions 
(Continuation) 


Run 

Result 

DiSt’n 

Mean 

Std  Dev 

Sample 

Seed 

46 

c 

* 

beta 

0.47 

0. 170 

800 

1 

47 

c 

400 

48 

c 

K 

200 

49 

c 

weibull 

800 

50 

c 

51 

c 

K 

200 

52 

c 

c 

beta 

0.47 

0.270 

800 

53 

c 

c 

400 

54 

c 

c 

200 

55 

c 

c 

beta 

0.75 

0.170 

800 

56 

c 

c 

400 

57 

c 

c 

200 

The  error  rates  for  the  two  attributes  are:  1/57  (<  2  percent),  for  modes, 
and  13/57  (23  percent)  for  stochastic  components.  The  error  rate  for  the 
latter  attribute  is  seen  to  be  larger  than  that  estimated  in  the  preliminary 
tests  in  which  beta  mixtures  were  used  exclusively. 
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TABLE  A-2.  Results  of  Numerical  Experiments  in  Testing  Neural 
Networks  With  Unimodal  and  Bimodal ,  Two-Component 
Distributions 


Component  1  Component  2 

Run  Modes  Result  Dist’n  Mix  Mean  S  D  Mean  S  D  Sample  Seed 


1 

2 

c 

c 

normal 

0.5 

0.35 

0.10 

0.65 

0.10 

800 

2 

c 

c 

400 

3 

c 

c 

200 

4 

2 

c 

c 

0.7 

0.35 

0.10 

0.65 

0.10 

800 

5 

c 

c 

400 

5 

c 

c 

200 

7 

2 

c 

c 

logistic 

0.7 

0.35 

0.10 

0.65 

0.10 

800 

8 

c 

c 

400 

9 

c 

c 

200 

10 

2 

[a] 

K 

c 

0.5 

0.35 

0.15 

0.65 

0.15 

800 

11 

X 

c 

400 

12 

« 

c 

200 

13 

1 

c 

« 

0.6 

0.35 

0.20 

0.65 

0.20 

800 

14 

c 

« 

400 

15 

c 

X 

200 

16 

2 

c 

c 

betaCb  ] 

0.5 

0.35 

0.09 

0.65 

0.09 

800 

17 

c 

c 

400 

18 

c 

c 

200 

19 

1 

c 

c 

logistic 

0.6 

0.35 

0.20 

0.65 

0.20 

400 

20 

c 

c 

200 

21 

2 

c 

c 

beta 

0.5 

0.35 

0.097 

0.65 

0.097 

800 

22 

c 

c 

400 

23 

c 

c 

200 

24 

2 

c 

c 

0.5 

0,35 

0.106 

0.65 

0.106 

800 

25 

c 

c 

400 

26 

c 

c 

200 

27 

2 

[a] 

c 

c 

0.6 

0.35 

0.106 

0.65 

0.106 

800 

28 

K 

c 

400 

29 

c 

c 

200 

30 

2 

Cal 

c 

c 

0.4 

0.35 

0.106 

0.65 

0.106 

800 

31 

c 

c 

400 

32 

c 

c 

200 

33 

1 

c 

c 

0.5 

0.35 

0.117 

0.65 

0.117 

800 

34 

c 

c 

400 

35 

c 

c 

200 

36 

1 

c 

c 

0.5 

0.35 

0.132 

n.65 

0.132 

800 

37 

c 

c 

400 

38 

c 

c 

200 

Continued  on  next  page. 


[a] 

[b] 


Histogram  is  distinctly  unimodal,  altho  density  is  bimodal. 
Threshold  parameter  of  first  beta  component  is  0  with  upper 
limit  of  0.7.  Threshold  of  second  beta  component  is  0.3  with 
upper  limit  of  1.0. 


TABLE  A-2.  Results  of  Numerical  Experiments  in  Testing  Neural 
Networks  With  Unimodal  and  Bimodal ,  Two-Component 
Distributions  (Continuation) 


Run 

Modes 

Result  Dist'n 

Mix 

Component  1 
Mean  S  D 

Component  2 

Mean  S  D  Sample  Seed 

39 

1 

c 

c 

0.5 

0.35 

0.156 

0.65 

0.156 

800 

40 

c 

c 

400 

41 

1 

c 

c 

0.6 

0.35 

0.117 

0.65 

0.117 

800 

42 

c 

c 

4u  • 

43 

c 

c 

200 

44 

1 

c 

c 

0.7 

0.35 

0.117 

0.65 

0.117 

800 

45 

c 

c 

400 

46 

c 

c 

200 

47 

2 

c 

c  logistic 

0.7 

0.35 

0.117 

0.65 

0. 1 17 

800 

48 

c 

c 

400 

49 

c 

c 

200 

50 

2  [a] 

x 

X 

0.5 

0.35 

0.156 

0.65 

0.156 

800 

51 

K 

c 

400 

52 

K 

200 

53 

1 

c 

K 

0.5 

0.35 

0.200 

0.65 

0.200 

800 

54 

c 

400 

55 

c 

* 

200 

[a]  Two  modes  are  barely  evident  in  the  density,  but  are  not  clearly  evident 
in  a  typical  histogram. 

Neural-net  error  rates  for  the  two-component  distributions  tested  are  7/55 
(13  percent),  for  identifying  modes,  and  8/55  (14  percent)  for  identifying 
stochastic  couponents . 

4.  Performance  Summary  and  Conclusions 

In  total  112  data  sets  were  used  to  challenge  the  identification  accuracy 
of  these  two  neural  networks.  The  net  which  identifies  modality  of  the  popu¬ 
lation  density  was  correct  in  93  percent  (104/112)  of  the  sets.  The  net  for 
identifying  stochastic  components  did  not  perform  quite  as  well,  yielding  81 
percent  (91/112J  correct  responses.  Of  course,  these  results  depend  strongly 
upon  the  choice  of  distributions  tested  and  to  a  lesser  extent  upon  the  sample 
of  data  drawn  from  each.  The  performance  of  the  net  for  identifying  modality 
was  shown  to  be  better  than  that  for  an  alternative,  statistically  based 
method.  There  is  no  known  alternative  for  the  network  which  identifies 
stochastic  components. 
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APPENDIX  B 


Neural  Networks  for  a 
Statistical  Advisor  System 

Simscript  Computer  Source  Program 
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Neural  Networks  for  a  Statistical  Advisor  System 
Simscript  Computer  Source  Program 


1  PREAMBLE  ’’RUN.1D.NET 

2  NORMALLY  MODE  IS  REAL. 

3  DEFINE  SIGMOID  AS  A  REAL  FUNCTION  GIVEN  2  ARGUMENTS 

4  END  ’’PREAMBLE 


MAIN  ’’RUN. 1D.NET 


3 

4 

5 
C 

7 

8 
9 

10 

11 

12 

13 

14 

15 
15 


DRIVER  FOR  THE  ROUTINE  N1D.NET  FOR  ANALYSIS  OF  UNIVARIATE  DATA. 
Program  has  been  augmented  to  read  a  data  file  of  random  variables, 
create  a  histogram  and  normalize  it  for  the  neural  net,  and  call  the 
program  N1D.NET  with  appropriate  input  vector. 

Required  Programs/Functions;  DYQUANT,  UPDT.HIST,  SIGMOID. 

DEFINE  ANSWER, FI  LI  NAM. TITLE  AS  TEXT  VARIABLES 

DEFINE  FLAGN.FLAGP.I ,ITRUNC.J.N,NCELLS.NCOUNT,NDATA  AS  INTEGER 

VARIABLES 

DEFINE  HISTV.NXV  AS  INTEGER.  1 -DIMENSIONAL  ARRAYS 
RESERVE  NXV(»)  AS  5 

DEFINE  NODEV.QV.XINV  AS  REAL.  1 -DIMENSIONAL  ARRAYS 
RESERVE  QV(»)  AS  5  ’’MARKERS  FOR  QUANTILES  OF  XINV 
LET  NCELLS=20 


17  LET  LSDL-3.0 

18  LET  USDL=4.5 

19  LET  WLIM=l.l 

20  RESERVE  HISTV(*) .N0DEV(*)  AS  NCELLS 

21  LET  LINES. V=9999 

22  LET  FILINAM  =  'RV.DATA' 

23  ’LO’ PRINT  1  LINE  WITH  FILINAM 

24  THUS 

INPUT  THE  NUMBER  OF  DATA  POINTS  TO  BE  READ  FROM  »»«*»»»» 

36  READ  NDATA 

37  RESERVE  XINV(»)  AS  NDATA 
28  ’  ’ 

29  ’’OPEN  UNIT  4  FOR  INPUT  OF  THE  RANDOM- VARIABLE  DATA. 

30  ’  ’ 

31  OPEN  UNIT  4  FOR  INPUT. 

32  OLD, 

33  FILE  NAME  IS  FILINAM 

34  RECORD  SIZE  IS  80 

35  USE  UNIT  4  FOR  INPUT 

36  READ  TITLE  USING  UNIT  4 

37  FOR  I-i  TO  NDATA,  READ  XINV(I)  USING  UNIT  4 

38  CLOSE  UNIT  4  ’’FOR  THIS  INPUT 

39  ’  ’ 

40  ’’GET  AVGX,  STDV,  XMIN,  AND  XMAX. 

4!  ” 
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42  LET  XMIN=RINF.C 

43  LET  XMAX=  -RINF.C 

44  LET  AVGX=0.0 

45  LET  STDV=0.0 

46  FOR  1=1  TO  NDATA  DO 

47  LET  X=XINV(I) 

48  CALL  DYQUANT{0.5,()V(»)  ,I,X.NXV(«)) 

49  LET  XMIN=MIN.F(XMIN,X) 

50  LET  XMAX=MAX.F(XMAX,X) 

51  ADD  X  TO  AVGX 

52  ADD  X»»2  TO  STDV 

53  LOOP  ’’OVER  (I)  POINTS 

54  LET  AVGX= AVGX/ NDATA 

55  LET  STDV=(STDV  -  NDATA»AVGX»»2) / (NDATA  -  1) 

56  LET  STDV=SQRT.F(STDV) 

57  IF  QV(l)  NE  XMIN  OR  QV(5)  NE  XMAX 

58  PRINT  2  LINES  WITH  QV(l) , XMIN, QV(5) , XMAX 

59  THUS 

ERROR.  INCONSISTENCY  IN  CALCULATING  MIN  AND  MAX.  First  of  pair  obtained 
in  routine  DYQUANT:  (  »»».»«)()(,  »»*.*»««)  and  (  »««.»»»«,  *»«,»»»«). 

62  STOP 

63  OTHERWISE 

64  SKIP  1  LINE 

65  PRINT  4  LINES  WITH  XMIN. XMAX, AVGX, STDV, QV(2) ,QV(3) ,QV(4) 

66  THUS 

Min  X  =  »»«».««»«  Max  X  =  *»««.»»*» 

Avg  X  =  *»»)(.»«»«  S  D  X  =  ««««.«#»» 

X.25  =  »»»*.*»»»  X.50  =  »»»».»*»»  X.75  =  »»»«.»»»» 

71  IF  XMIN  LE  0.0 

72  LET  FLAGN=1  "INDICATING  THAT  RV  CAN  BE  NEGATIVE 

73  LET  FLAGP=0  "INDICATING  THAT  RV  IS  NOT  A  PROPORTION 

74  GO  TO  JO 

75  OTHERWISE 

76  PRINT  2  LINES  THUS 

CAN  THE  RANDOM  VARIABLE  TAKE  ON  NEGATIVE  VALUES  OR  BE  TREATED  AS 
EFFECTIVELY  UNBOUNDED  FROM  BELOW?  (YES  OR  NO) . 

79  READ  ANSWER 

80  IF  SUBSTR.F( ANSWER, 1.1)  =  'Y'  OR  SUBSTR. F (ANSWER . 1 . 1 )  =  ‘y’ 

81  LET  FLAGN=1 

82  LET  FLAGP=0 

83  GO  TO  JO 

84  OTHERWISE 

85  LET  FLAGN=0 

86  IF  XMAX  <  1.0 

87  PRINT  2  LINES  THUS 

DOES  THE  RANDOM  VARIABLE  REPRESENT  A  PROPORTION  (PROBABILITY)?  (Y  OR  N) . 
That  is,  must  the  random  variable  lie  within  the  range  (0,  1)? 

90  READ  ANSWER 

91  IF  SUBSTR. F( ANSWER, 1,1)  =  'T  OR  SUBSTR. F(ANSWER, 1 , 1)  =  'y' 

92  LET  FLAGP=1 

93  GO  TO  JO 

94  OTHERWISE 

95  ALWAYS 
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LET  FLAGP=0 


96 

97  ’  ■ 

98  "DETERMINE  HISTOGRAM  LIMITS. 

99  '  ’ 

100  ’JO’ IF  FLAGP=1 

101  LET  XHM1N=0.0 

102  LET  XHMAX=1.0 

103  GO  TO  J1 

104  OTHERWISE 

105  IF  FLAGN=0  ’’X  IS  STRICTLY  POSITIVE 

106  IF  AVGX  -  USDL»STDV  >0.0 

107  ’  ’ 

108  ’’TREAT  X  AS  EFFECTIVELY  UNBOUNDED  IN  CALCULATING  HISTO  LIMITS. 

109  ’  ’ 

no  GO  TO  J3 

111  OTHERWISE 

112  LET  XHMINO=AVGX  -  LSDL»STDV 

113  LET  XHHIN1=XMIN  -  0.5*3TDV 

114  LET  XHMIN2=XMIN  -  0.2»(QV(2)  -  XMIN) 

115  LET  XHMIN=MAX.F(0.0,XHMIN0,XHMIN1,XHMIN2) 

116  LET  XHMAX=AVGX  +  USDL»STDV 

117  IF  XHMAX  <  XMAX 

118  LET  XHMAX=XHMAX  +  WLIMo (XMAX-XHMAX) 

119  ALWAYS 

120  ’  ’ 

121  ’’CONSTRAINTS  ON  THE  LIMITS. 

122  ’  ’ 

123  LET  XHMAX1=XMAX  +  0.5«STDV 

124  LET  XHMAX2=XMAX  +  0.2»(XMAX  -  QV(4)) 

125  LET  XHMAX= MI N.F( XHMAX, XHMAX 1,XHMAX2) 

126  GO  TO  J1 

127  OTHERWISE  ’’RANGE  OF  X  INCLUDES  NEGATIVES  OR  IS  EFFECTIVELY  UNBOUNDED 

128  ’J3’LET  XHMIN=AVGX  -  USDL*STDV 

129  IF  XMIN  <  XHMIN 

130  LET  XHMIN=XHMIN  -  WLIM« (XHMIN-XMIN) 

131  ALWAYS 

132  LET  XHMAX=AVGX  +  USDL»STDV 

133  IF  XMAX  >  XHMAX 

134  LET  XHMAX=XHMAX  +  WLIM» (XMAX-XHMAX) 

135  ALWAYS 

136  ’  ’ 

137  ’ ’CONSTRAINTS  ON  THE  LIMITS. 

138  ’  ’ 

139  LET  XHMIN1=XMIN  -  0.5»STDV 

140  LET  XHMIN2=XMIN  -  0.2»((3V(2)  -  XMIN) 

141  LET  XHMIN=MAX.F(XHMIN,XHMIN1,XHMIN2) 

142  LET  XHMAX1=XMAX  +  0.5»STDV 

143  LET  XHMAX2=XMAX  +  0.2)t(XMAX  -  QV(4)) 

144  LET  XHMAX= MI N.F( XHMAX, XHMAX 1.XHMAX2) 

145  'Jl’LET  DELX= (XHMAX  -  XHMIN) /NCELLS 

146  IF  DELX  LE  0.0 

147  PRINT  1  LINE  THUS 

TROUBLE  WITH  INPUT  DATA.  Calculated  DELX  in  histogram  is  0. 

149  STOP 


150  OTHERWISE 

151  LET  ITRUNC=0 

152  LET  NC0UNT=0 

153  FOR  1=1  TO  NDATA  DO 

154  LET  X=XINV(I) 

155  ’  ’ 

156  "LOCATE  VALUES  OF  X  WITH  RESF  TO  HISTO  CELLS  AND  INCREMENT  CELL  FREQS. 

157  " 

158  CALL  UPDT.HIST(X,NCELLS,ITRUNC,XHMIN,DELX,HISTV(»))  YIELDING  N 

159  ADD  N  TO  NCOUNT 

160  LOOP  "OVER  (I)  POINTS 

161  IF  FLAGP=1 

162  GO  TO  J4 

163  OTHERWISE 

164  ‘  ’ 

165  "CHECK  FOR  EXCEPTIONAL  COUNTS  IN  LAST  CELL. 

166  ’  ’ 

167  IF  2»HISTV(NCELLS)  >  HISTV(NCELLS-l)  +  HISTV(NCELLS-2)  AND  FLAGP  NE  1 

168  ■  ’ 

169  "UPPER-TAIL  OUTLIERS  MAY  BE  PRESENT.  TRUNCATE  VALUES  ABOVE  XHMAX. 

170  ’  ' 

171  LET  ITRUNC=1 

172  LET  NC0UNT=0 

173  FOR  1=1  TO  NCELLS,  LET  HISTV(I)=0 

174  FOR  1=1  TO  NDATA  DO 

175  LET  X=XINV(I) 

176  CALL  UPDT.HIST(X,NCELI,S,ITRUNC,XHMIN,DELX,HISTV(*))  YIELDING  N 

177  ADD  N  TO  NCOUNT 

178  LOOP  "OVER  (I)  DATA  POINTS 

179  ’  ’ 

180  "TRANSFORM  HISTO  FREQUENCIES  TO  GET  NODAL  VALUES  FOR  A  NEURAL  NET, 

181  ’  ’ 

182  ALWAYS 

183  'J4’IF  NCOUNT  LE  0 

184  PRINT  1  LINE  THUS 

TROUBLE  IN  GETTING  HISTOGRAM  COUNT.  TOTAL  COUNT  =  0. 

186  STOP 

187  OTHERWISE 

188  FOR  1=1  TO  NCELLS  DO 

189  LET  NODEV(I)=HISTV( I) /NCOUNT  +  0.1 

190  IF  NOD’^Vd)  >  0.9 

191  PRINT  2  LINES  WITH  I.NODEVd) 

192  THUS 

TROUBLE.  Error  in  normalizing  for  neural-net  input  nodes. 

NODEV(**)  =  ».*«»» 

195  STOP 

196  OTHERWISE 

197  LOOP  "OVER  (1)  HISTOGRAM  CELLS 

198  ’  ' 

199  "DETERMINE  WHETHER  DENSITY  OF  X  IS  UNIMODAL  OR  MULTIMODAL. 

200  ’  ’ 

201  PRINT  3  LINES  WITH  T I TLE.XHMIN, XHMAX, NCOUNT, NDATA 

202  THUS 
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SUBJECT: 

Min  X  in  Histogram:  »».«»»*  Max  X  in  Histogran:  «».»*»» 
Total  Frequency  in  Histogram:  *»»»  Total  RVs  Generated:  »»*» 

206  CALL  N1D.NET  GIVEN  1,N0DEV(»)  YIELDING  UMOUT 

207  PRINT  1  LINE  WITH  UMOUT 

208  THUS 


Output  node  for  modality  of  density  is  »».«»«» 

210  ’  ’ 

211  "DETERMINE  WHETHER  DENSITY  REPRESENTS  A  MIXTURE  MODEL. 

212  ’  ’ 

213  SKIP  1  LINE 

214  CALL  N1D.NET  GIVEN  2,N0DEV(»)  YIELDING  UMOUT 

215  PRINT  1  LINE  WITH  UMOUT 

216  THUS 


Output 

218 

219  ’  ’ 

220  ’  ’ 
221  " 
222  ’  ’ 

223  ’  ’ 

224  ’  ’ 

225  '  ’ 

226 

227  END 


node  for  stochastic  components  is  ».*»»« 

SKIP  1  LINE 

PRINT  1  LINE  THUS 

OTHER  INPUTS  WANTED’  (Y  OR  N) . 

READ  ANSWER 

IF  SUBSTR.F( ANSWER, 1.1)  =  ‘Y’  OR  SUBSTR.F(ANSWER, 1 . 1) 
RELEASE  XINV(») 

GO  TO  LO 
OTHERWISE 
STOP 

’ ’RUN.1D.NET 


■y' 


1 

2 

3 

4 

5 

6 

7 

8 
9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 
21 
22 

23 

24 

25 

26 

27 

28 


ROUTINE  N1D.NET  GIVEN  lA.  XINV  YIELDING  UMOUT 

t  » 

’’Program  calculates  attributes  of  a  one-dimensional  density  function  from 
’’Nl-cell  histogram  (scaled),  using  a  neural  network.  Input  layer  of  the 
’’network  contains  N1  nodes;  the  middle  layer  contains  N2  nodes;  and  the 
’’output  layer  contains  one  node.  Nodal  architecture  permits  each  input 
’’node  to  be  connected  to  each  node  of  the  middle  layer.  The  output  node 
’’is,  in  turn,  connected  to  each  node  of  the  middle  layer.  The  attribute 
’’represented  by  the  output  nodal  value  UMOUT  depends  upon  the  value  of 
’’the  integer  flag  lA.  If  lA  =  1,  the  output  encodes  modality  of  the 
’’probability  denSity-O. 1 ,  for  unimodal  and  0.9  otherwise.  If  lA  =  2, 
’’the  output  node  encodes  number  of  stochastic  components  in  density 
’’mixture  model--0.1  for  one  component  and  0.9  otherwise.  Input  nodal 
’ ’values  are  contained  in  vector  XINV,  representing  Scaled  frequencies 
’’in  a  histogram  of  the  data.  Use  of  program  for  data  samples  less  than 
’’about  150  is  not  recommended.  The  two-argument  SIGMOID  function  is 
’’required.  Also,  two  files  of  weights  are  necessary--NETS. WT2.DATA 
’ ’and  NETS. WT4. DATA. 

I  $ 

DEFINE  ANSWER.FILWNAM  AS  TEXT  VARIABLES 

DEFINE  I, IA,IPRINT.J,K,L, LAYERS, M,N,NWTS  AS  INTEGER  VARIABLES 

DEFINE  NV  AS  AN  INTEGER.  1 -DIMENSIONAL  ARRAY 

DEFINE  PARMV.XINV  AS  REAL.  1 -DIMENSIONAL  ARRAYS 

DEFINE  XA  AS  A  REAL.  2-DIMENSIONAL  ARRAY 

DEFINE  WTA  AS  A  REAL,  3 -DIMENSIONAL  ARRAY 

t  t 

"DEFINITIONS  OF  ARRAYS: 

»  I 
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XINV(l)  _  VALUE  OF  THE  Uh  NODE  IN  THE  INPUT  LAYER  (NUMBER  1). 

XA(K.I)  VALUE  OF  THE  Ith  NODE  IN  LAYER  K.  K=1  (input)  TO  LAYERS. 

WTA(K,I,jT"  WEIGHT  ON  ARC  BETW  Uh  NODE  OF  LAYER  K+1  AND  Jth  NODE  OF 
LAYER  K,  FOR  K=1  TO  LAYERS-1,  1=1  TO  NV(X+1),J=1  TO  NV(K) 


DEFAULT  VALUES. 


THE  FOLLOWING  FILES  CONTAIN  WEIGHTS  FOR  1-D  NETWORKS  OF  3  LAYERS  EACH 
WITH  LAYER  1  HAVING  20  INPUT  NODES.  LAYER  2  (MIDDLE  LAYER) 

CONTAINING  7  NODES,  AND  LAYER  3  CONTAINING  A  SINGLE  OUTPUT  NODE. 

NETS . WT2 . DATA  _  to  determine  number  of  stochastic  components.  This 
weight  set  has  proven  successful  in  all  instances  of 
unimodal  data  with  one  component.  In  those  instances  of 
challenge  sets  of  bimodal  data  with  two  components,  the 
weights  yield  good  predictability.  In  the  case  of  uni¬ 
modal  densities  with  two  components,  the  weights  have 
been  less  successful  than  in  the  other  cases. 

NETS . WT4 . DATA  _  to  determine  modality  of  the  population  density.  If 
value  of  the  output  node  exceeds  0.5,  a  multimodal 
density  is  implied;  otherwise,  density  is  predicted  as 
unimodal . 


51  ■  ’ 

52  LET  IPRINT=0  "FOR  NO  PRINT 

53  LET  LAYERS=3 

54  RESERVE  NV(»)  AS  LAYERS 

55  RESERVE  PARMV(»)  AS  4  "PARAMETERS  OF  THE  SIGMOID  FUNCTION 

56  LET  PARMV(l)  =0.0  "LOCATION 

57  LET  PARMV(2)  =  1.0  "SCALE 

58  LET  PARMV(3)=0.0  "MIN  VALUE 

59  LET  PARMV(4)  =  1.0  "MAX  VALUE 

60  RESERVE  XA(»,»)  AS  LAYERS  BY  » 

61  RESERVE  WTA (»,*,*)  AS  LAYERS-1  BY  * 

62  LET  NV(1)=20  "DEFAULT 

63  LET  NV(2)=7  "DEFAULT 

64  LET  NV(3)  =  1  "DEFAULT 

65  '  ’ 

66  "CHECK  FOR  VALID  INPUTS. 

67  '  ' 

68  IF  lA  LE  0  OR  lA  >  2 

69  PRINT  1  LINE  WITH  lA 

70  THUS 

INVALID  INPUT  TO  ROUTINE  N1D.NET.  lA  = 

72  STOP 

73  OTHERWISE 

74  IF  DIM.F(XINV(»))  NE  20 

75  PRINT  1  LINE  THUS 

INVALID  DIMENSION  OF  THE  INPUT  VECTOR  IN  ROUTINE  N1D.NET. 

77  STOP 

78  OTHERWISE 

79  IF  IA=1 

80  LET  FILWNAM  =  'NETS .WT4 .DATA* 

81  OTHERWISE 
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82  LET  FILWNAM  =  ‘NETS. WT2.DATA' 

83  ALWAYS 

84  ’  ’ 

85  "RESERVE  ARRAYS. 

86  ’  ’ 

87  FOR  K=1  TO  LAYERS,  RESERVE  XA(K,»)  AS  NV(K) 

88  FOR  K=1  TO  LAYERS- 1,  RESERVE  WTA(K,»,»)  AS  NV(K+1)  BY  NV(K) 

89  •  ’ 

90  "MOVE  INPUT  NODAL  VALUES  TO  XA. 

91  ’  ’ 

92  FOR  1=1  TO  NV(l),  LET  XA(1 .1) =XINV{I) 

93  '  ’ 

94  "OPEN  UNIT  4  FOR  NETWORK  WEIGHTS. 

95  ’  ’ 

96  OPEN  UNIT  4  FOR  INPUT, 

97  OLD, 

98  FILE  NAME  IS  FILWNAM 

99  RECORD  SIZE  IS  80 

100  USE  UNIT  4  FOR  INPUT 

101  LET  NWTS=0 

102  FOR  K=1  TO  LAYERS- 1  DO 

103  LET  M=NV(K+1) 

104  LET  N=NV(K) 

105  ADD  M»N  TO  NWTS 

.106  FOR  1  =  1  TO  M,  FOR  J=1  TO  N,  READ  WTA(K,I,J)  USING  UNIT  4 

107  LOOP  "OVER  (K)  LAYERS 

108  CLOSE  UNIT  4 

109  IF  IPRINT=1 

no  SKIP  2  LINES 

111  PRINT  5  LINES  WITH  FILWNAM, LAYERS, NWTS 

112  THUS 

File  name  for  network  weights: 

NETWORK  ARCHITECTURE: 

Total  Layers  in  Network:  *» 

Total  Number  of  Weights:  «»»»«* 

118  ALWAYS 

119  FOR  K=1  TO  LAYERS- 1  DO 

120  LET  M=NV<K+1) 

121  LET  N=NV(K) 

122  FOR  1=1  TO  M  DO 

123  LET  SUM=0.0 

124  FOR  J=1  TO  N  DO 

125  ADD  WrA(K,I,J)»XA(K,J)  TO  SUM 

126  LOOP  "OVER  (J)  BOTTOM  NODES 

127  LET  XA(K+l,I)=SIGMOID(SUM,PARMV(«))  "K+ 1ST  LAYER  NODES 

128  LOOP  "OVER  TOP  NODES 

129  LOOP  "OVER  (K)  LAYERS 

130  LET  UMOUT=XA (LAYERS,!) 

131  ’  ’ 

132  "PRINT  OUTPUT. 

133  ’  ’ 

134  IF  IPRINT=1 

135  IF  IA=1 
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136  LET  ANSWER  =  ’modality  of  density" 

137  OTHERWISE 

138  LET  ANSWER  =  ’stochastic  components’  • 

139  ALWAYS 

140  SKIP  1  LINE 

141  PRINT  1  LINE  WITH  ANSWER, UMOOT 

142  THUS 

Output  node  for  *»»»»»»»»»»»»»«»*»«»»«  =  »».««»» 

144  ALWAYS 

145  RELEASE  PARMV(») 

146  RELEASE  NV(») 

147  RELEASE  XA(»,») 

148  RELEASE  WTA(»,»,») 

149  RETURN 

150  END  "NID-NET 
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FUNCTION  SIGMOID  (X,  PARMV) 

»  I 

''Function  calculates  the  sigmoidal  value  with  argument  X  and  parameter 
''vector  PARMV.  There  are  four  elements  in  PARMV(«):  (1)  bha  location 
''or  bias  parameter  for  X,  (2)  scale  parameter  for  X,  (3)  min  value  of 
''the  function,  and  (4)  max  value  of  the  function.  The  function,  f (x) , 

' ' is  given  by : 

I  » 

"  (PARMV (4) -PARMV (3)  )/(l  +  exp  -[  (X-PARMV(l) ) /PARMV{2)  ])  +  PARMVO)  . 

I  I 

''The  function  may  be  used  as  a  smooth  limiter  for  variable  X,  taking  on 
"value  PARMV(3)  +  (PARMV(4) -PARMV(3) ) /2  when  i  -  (X-PARMV( 1) ) /PARMV(2) 
''is  zero,  and  approaching  PARMV(3)  as  z  approaches  -inf  and  PARMV(4) 
''as  X  approaches  +inf. 

''Note  that  f(z)  is  quite  linear  over  the  region  -0.5  <  z  <  0.5.  A 
''noticeable  nonlinearity  develops  for  abs(z)  >  0.7,  approximately. 

f  I 

DEFINE  FLAGU  AS  AN  INTEGER  VARIABLE 
DEFINE  PARMV  AS  A  REAL,  1-DIMENSIONAL  ARRAY 
LET  FLAGU=1 


21  IF  FLAGU=1 

22  RETURN  WITH  1.0/(1.0  ♦  EXP.F(-X)) 

23  OTHERWISE 

24  IF  PARMV(2)  LE  0.0 

25  PRINT  1  LINE  THUS 

INPUT  ERROR  TO  FUNCTION  SIGMOID.  SCALE  PARAMETER  IS  L.E.  0. 
27  STOP 


28  OTHERWISE 

29  LET  Z=(X  -  PARMV(1))/PARMV(2) 

30  RETURN  WITH  PARMV{3)  +  (PARMV(4)  -  PABMV(3) ) / ( 1 . 0  +  EXP.F(-Z)) 

31  END  "FUNCTION  SIGMOID 
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ROUTINE  UPDT.HIST{X,NCELLS,ITRUNC,XMIN,DELX,HISTV)  YIELDING  NCOUNT 

t  I 

''Routine  updates  frequency  count  in  the  histogram  HISTV  to  include  the 
''random  variable  X.  Number  of  cells  in  the  histogram  is  NCELLS,  The 
''min  (lower-limit)  of  the  independent  variable  is  XMIN  and  cell  width 
''or  independent-variable  increment  is  DELX.  Independent  variable  is  con- 
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’’sidered  continuous  and  HISTV  is  discrete.  The  vector  HISTV(»)  is 
’’dimensioned  NCELLS.  A  cumulative  counter  NCOUNT  is  increased  by  1  if 
’’either  of  these  conditions  holds:  (a)  flag  ITRUNC  is  set  to  0 ,  or  (b) 
’’J{  le  maximum  cell  upper  bound,  XMIN  +  NCELLS»DELX. 

I  I 

DEFINE  ITRUNC. K, NCELLS. NCOUNT  AS  INTEGER  VARIABLES 
DEFINE  HISTV  AS  AN  INTEGER.  1-DIMENSIONAL  ARRAY 
FOR  K=1  TO  NCELLS  DO 

LET  XUP=XMIN  +  DELX»K 
IF  X  LE  XUP 

ADD  1  TO  HISTV(K) 

ADD  1  TO  NCOUNT 
RETURN 
OTHERWISE 

LOOP  "OVER  (K)  HISTOGRAM  CELLS 
IF  ITRUNC=0 

ADD  1  TO  HISTV (NCELLS) 

ADD  1  TO  NCOUNT 
ALWAYS 
RETURN 

END  ”0F  ROUTINE  UPDT.HIST 
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ROUTINE  DYQUANT  (P.  QV.  N.  XN,  NXV) 

t  • 

’’Program  calculates  the  P-quantile  from  a  set  of  observations  presented 
’’dynamically  (as  generated).  Quantile  is  updated  after  each  observation 
’’is  made.  The  number  (N)  of  newest  observation  and  its  value  (XN)  are 
’’specified  by  the  calling  program.  Ref  to  method  is  made  to  an  article 
’’in  Communications  of  the  ACM,  Vol  28,  No  10,  Oct  1985:  Raj  Jain  and 
’’Imrich  Chlamatac,  ‘The  P-Squared  Algorithm  for  Dynamic  Calculation  of 
’’Quantiles  and  Histograms  'Without  Storing  Observations’,  p.  1076  ff.  A 
’’set  of  five  'markers*  of  RV  quantiles  are  stored  in  QV(»)  and  a  set  of 
’’five  ’horizontal  positions’  for  markers  are  stored  in  NXV(*).  Markers 
’’are  used  to  locate  the  real-valued  quantiles.  These  are  defined  as: 

’’QV(l)  _  minimum  value  over  all  RV  observations  currently  made. 

'’QV(2)  _  lowest  middle  marker,  corresponding  to  the  P/2  quantile. 

’’QV(3)  _  desired  P  quantile. 

’’QV(4)  _  highest  middle  marker,  corresponding  to  (1  +  P)/2  quantile. 

'’QV(5)  _  maximum  value  over  all  RV  observations  currently  made. 

I  I 

' 'Routine  is  called  after  each  observation  is  made  with  the  number  and 
’’value  of  the  observation.  First  five  observations  are  sorted  in  ascend- 
’’ing  order  and  placed  in  QV(»).  Next  (Nth)  observation  is  given  in  XN. 
’’A  vector  of  ’horizontal’  marker  positions,  NXV(») ,  is  maintained  by  the 
’’routine.  After  the  Nth  observation,  NXV(i)  =  number  of  observations 
’’(approx)  whose  value  is  l.e.  QV(i) ,  i  =  1,  ....  5. 

f  » 

DEFINE  I,J,K,N  AS  INTEGER  VARIABLES 
DEFINE  QV  AS  A  REAL,  1 -DIMENSIONAL  ARRAY 
DEFINE  DNV  .AS  A  REAL,  S.AVED,  l-DI.MEKSION.AL  ARRAY 
RESERVE  DNV(»)  AS  5 

DEFINE  NXV  AS  AN  INTEGER,  I -DIMENSIONAL  ARRAY 
IF  N  >  5 

GO  TO  LO 


B-10 


33 

34 

35 

36 

37 

38 

39 

40 

41 

42 

43 

44 

45 

46 

47 

48 

49 

50 

51 

52 

53 

54 

55 

56 

57 

58 

59 

60 
61 
62 

63 

64 

65 

66 

67 

68 

69 

70 

71 

72 

73 

74 

75 

76 

77 

78 

79 

80 
81 
82 
8o 

84 

85 

86 


OTHERWISE 
IF  N=1 

LET  QV(1)=XN 

FOR  1=1  TO  5,  LET  NXV(I)=I 
LET  DNV{1)=1.0 
RETURN 
OTHERWISE 
IF  N=2 

IF  XN  >  QV(1) 

LET  QV(2)=XM 
OTHERWISE 

LET  QV(2)=QV(1) 

LET  QV(1)=XN 
ALWAYS 
RETURN 
OTHERWISE 
IF  N=3 

IF  XN  <  QV(l) 

LET  QV(3)=QV(2) 

LET  QV(2)=QV(1) 

LET  QV(1)=XN 
RETURN 
OTHERWISE 
IF  XN  >  QV(2) 

LET  QV{3)=XN 
RETURN 
OTHERWISE 
LET  QV{3)=QV(2) 

LET  QV(2)=XN 
RETURN 
OTHERWISE 
IF  N=4 

IF  XN  <  QV(1) 

LET  QV(4)=QV(3) 

LET  QV(3)=QV(2) 

LET  QV(2)=QV(1) 

LET  QV(1)=XN 
RETURN 
OTHERWISE 
IF  XN  >  (}V(3) 

LET  QV(4)=XN 
RETURN 
OTHERWISE 
IF  XN  <  QV(2) 

LET  QV(4)=QV(3) 

LET  QV(3)=QV(2) 

LET  QV(2)=XN 
RETURN 
OTHERWISE 

ijC.1  vvi>»;=wvvo; 

LET  0V(3)=XN 
RETURN 

OTHERWISE  ’'N=5 
IF  XN  <  QV(1) 
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FOR  1=1  TO  4,  LET  QV{5-I+1) =07(5-1) 

LET  QV(1)=XM 
RETURN 
OTHERWISE 
IF  XN  >  QV(4) 

LET  QV(5)=XN 
RETURN 
OTHERWISE 
IF  XN  <  QV(2) 

FOR  1=1  TO  3,  LET  QV(5-I+1) =QV(5-I) 

LET  QV(2)=XN 
RETURN 
OTHERWISE 
IF  XN  <  QV(3) 

FOR  1=1  TO  2,  LET  QV(5-I+1) =QV(5-I) 

LET  QV(3)=XN 
RETURN 
OTHERWISE 
LET  QV(5)=QV(4) 

LET  QV(4)=XN 
RETURN 

>  t 

"CALCULATE  THE  DESIRED  MARKER  POSITIONS:  DN2,  DN3,  DN4. 

t  t 

'LO’LET  NM1=N-1 
LET  DNV(1)=1.0 
LET  DNV(2)=0.5»P)tNMl  +  1.0 
LET  DNV(3)=NM1»P+1.0 
LET  DNV(4)=0.5)K1,0+P)«NM1+1.0 
LET  DNV(5)=N 
IF  XN  <  QV(l) 

LET  QV(1)=XN 
LET  K=1 
GO  TO  LI 
OTHERWISE 
IF  XN  >  QV(5) 

LET  QV(5)=XN 
LET  K=4 
GO  TO  LI 
OTHERWISE 

"FIND  CELL  K  SUCH  THAT  QV(K)  LE  XN  <  QV(K+1) 

t  r 

FOR  K=1  TO  4  DO 

IF  QV(K)  LE  XN  AND  XN  <  QV(K+1) 

GO  TO  LI 
OTHERWISE 
LOOP  ’ ’OVER  K 
IF  XN=QV(5) 

LET  K=4 
ALWAYS 

t  t 

"INCREMENT  POSITIONS  OF  MARKERS  K+1  THRU  5. 
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’LTFOR  I=K+1  TO  5.  ADD  1  TO  NXVd) 

t  t 

’’ADJUST  HEIGHTS  OF  MARKERS  2  THRU  4. 

I  t 

FOR  1=2  TO  4  DO 

LET  J=0  ”T0  INDICATE  SIGN  OF  ADJUSTMENT 
LET  D=DNV(I)-NXV(I) 

IF  D  GE  1.0  AND  NXVd  +  l)  >  NXVd)  +  l 
LET  J=1 
GO  TO  L2 
OTHERWISE 

IF  D  LE  -1.0  AND  NXVd-1)  <  NXVd)-l 
LET  J=  -1 

’L2’  LET  QI=QVd)+J/(NXV(H-l)-NXVd-l))»((NXVd)-NXVd-l)+J)» 

(QVd  +  l)-QVd))/(NXVd  +  l) -NXVd)  )  +  (NXVd  +  l) -NXVd) -J)« 
(QVd) -QVd-1))/ (NXVd) -NXVd- 1))) 

t  I 

’’TRIAL  VALUE  OF  QVCI)  IS  QI. 

I  t 

IF  QVd-1)  <  QI  AND  QI  <  QVd  +  1) 

LET  QVd)=QI 

OTHERWISE  ’’USE  LINEAR  FORM  TO  ADJUST  QVd) 

LET  QVd)  =QVd)+J«(QVd+J) -QVd)  )/(NXVd+J) -NXVd)) 
ALWAYS 

ADD  J  TO  NXVd) 

ALWAYS 

LOOP  ’’OVER  I 
RETURN 

END  "ROUTINE  DYQUANT 
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Director 

U.S.  Army  Materiel  Systems  Analysis  Activity 
Aberdeen  Proving  Ground,  MD  21005-5071 

ATTN:  AMXSY-DA  (Mr.  E.  Atzinger) 


Director 

Ballistic  Research  Laboratory 
Aberdeen  Proving  Ground,  MD  21005-5066 


ATTN:  SLCBR-D 


DISTRIBUTION  LIST 


Director 

TRAC-FLVN 

Fort  Leavenworth,  KS  66027-5200 
ATTN:  ATRC-TD 


Director 
HQ  TRAC -LEE 

Fort  Lee.  VA  23801-6000 
ATTN;  ATRC-L 


Director 

TRAC-WSMR 

White  Sands  Missile  Range,  NM  88002-5502 
ATTN:  ATRC 


Director 

U.S.  Army  Concepts  Analysis  Activity 
8120  Woodmont  Avenue 
Bethesda,  MD  20814-2797 

ATTN:  CSCA-ZA 


DLA  Operations  Research  and  Economic  Analysis 
Management  Support  Office  (DLA-LO (DORO) ) 
c/o  Defense  General  Supply  Center 
Richmond,  VA  23297-5000 


Defense  Logistics  Agency 

Operations  Research  and  Economic  Analysis  Office  (DCSC-LO) 
Defense  Construction  Supply  Center 
3990  East  Broad  Street 
Columbus.  OH  43215-5000 


Defense  Logistics  Agency 

Operations  Research  and  Economic  Analysis  Office  (DESC-LO) 
Defense  Electronics  Supply  Center 
1507  Wilmington  Pike 
Dayton,  OH  45444-5000 


DISTRIBUTION  LIST 


Defense  Logistics  Agency 

Operations  Research  and  Economic  Analysis  Office  (DFSC-LO) 
Defense  Fuel  Supply  Center 
Building  8,  Cameron  Station 
Alexandria,  VA  22304-6160 


Defense  Logistics  Agency 

Operations  Research  Economic  Analyst  Office  (DGSC-LO) 
Defense  General  Supply  Center 
Richmond,  VA  23297-5000 


Defense  Logistics  Agency 

Operations  Research  and  Economic  Analysis  Office  (DTIC-LO) 
Defense  Technical  Information  Center 
Cameron  Station,  Room  5D146 
Alexandria,  VA  22304-6145 


Defense  Logistics  Agency 

Operations  Research  and  Economic  Analysis  Office  (DCASR-CHI-LO) 
O’Hare  International  Airport 
P.O.  Box  66475 
Chicago,  IL  60666-0475 


Defense  Logistics  Agency 

Operations  Research  and  Economic  Analysis  Office  (DISC-LO) 
Defense  Indust  lal  Supply  Center 
700  Robbins  Avenue 
Philadelphia,  PA  19111-5096 


